Real-Time Psychophysiological Emotional State Estimation in Digital Gameplay Scenarios

Despite the rising number of emotional state detection methods motivated by the popularity increase in affective computing techniques in recent years, they are yet faced with subject and domain transferability issues. In this paper, we present an improved methodology for modelling individuals’ emotional states in multimedia interactive environments. Our method relies on a two-layer classification process to classify Arousal and Valence based on four distinct physiological sensor inputs. The first classification layer uses several regression models to normalize each of the sensor inputs across participants and experimental conditions, while also correlating each input to either Arousal or Valence – effectively addressing the aforementioned transferability issues. The second classification layer then employs a residual sum of squares-based weighting scheme to merge the various regression outputs into one optimal Arousal/Valence classification in real-time, while maintaining a smooth prediction output. The presented method exhibited convincing accuracy ratings – 85% for Arousal and 78% for Valence –, which are only marginally worse than our previous non-real-time approach.